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SBIR Data Reporting / Graphing r equirements 
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> 



Need mterface to get'put information of school district, school region, school name, and class 

(with the option of retrieving this information directly from schools' student data files) 

Need interface to assign students to test versions and computer terminals 

Need to assign district, school region, school, class, and group It) and individual student ID 

numbers for storage and retrieval of data 

Display data as shown in the tables and graphs 

Display district, region, school, teacher, class, grade level on charts (see sample) 
Display name of test, CGI logo, copyright, disclaimer 

Use different colors for pre-test and post-test bars on graphs (maximum display of two (2) most 



MO) 



Need interface to assign ID number for storage and retrieval of data 

Display date of pre- and post- test, chronological age (date of test taken - date of birth), and grade 
of the student for individual student reports. Pre-test and post-test dates need to be labeled (see 
samples). 



SBIR Data Reports 

> There are three data report formats for class teachers to access and two more data report forms for 
school level and above. For individual parents, only individual student data summary is provided. 

Class Data Graphs (see the report samples on pages 4-7). 

Class Comparison with Multi-State Sample Average Tables (see the report samples on page 

Individual Student Data Summary (see the report samples on pages 10-12: This report is a 
two-section table to show students' perfonnance at their grade level and Full version level). 

District/Region/School/Class Data Graphs & Tables (see the report samples on pages 14-16). 
Click "region" at district level to link to the school level graphs and click "school" at region 
level will link to the class level graphs. Click pre-test/post-test bar will link to pre-test^post- 
test database tables. The database will have the function of retrieving all the (school/class) 
students' data with search and sort capabilities (e.g., search grade level = 1 aTid Blending 
Phonemes > = 3 ; sort total score in descendiTig nrHpr, etc.) The default fields for the table will 
be student name, 8 subtest total scores, total score for the test , and total time sp ent: 1 1 fields 
for pre-test and post-test respectively, {link interface design needs to be discussed \vith Noah) 

Comparison of Regipns/Schools/Classes to Multi-State Sample Average (see the report 
samples on pages 19-20) 

> Landscape will be the default print layout for all graphs. Portrait will be the default print layout for 
all tables. There should be an option of Landscape or Portrait for printing. 

> Teachers should be able to access a "first level button menu" from where they can click on the 
buttons to access the data they are interested and print the reports they need. The button's name 
reflects what information teachers can expect to get from diis report. When the pointer moves 
inside the button area, a floating window showing the detail information about the reports in this 
branch (e.g., Answer your questions: What is the profile of relative strengths & weakness for my 
class ...-see document Hyperlink Descriptions) . Reports are linked together. They can also be 
accessed by clicking on the buttons (hyperlinks) on another report. 

> There will be six buttons on the "first level button menu" screen. They are 
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Average Score on All Test Sections (Graph) 
Total Score Comparison (Graph) 

Class Comparison with Multi-State Sample Average (Table) 
Individual Student Data Summary (Table) 

District/Region/School/Class Data Graphs & Tables 

Comparison of Regions/Schools/Classes to Multi-State Sample Averages (Table) 



> For individual packages (packages for parents), only Individual Student Data Summary report is 
needed. 

> Class teachers can only access their students' data, so when they access the "first level button 
menu", buttons for the school and above levels are inactive. 

\v nrt :il T T» 1- 1 ^ A>f«^,, U,,^^«« «11 ^V.^ r^^n^Xsr, 0*1^ /^Ki-i-^^rw r\-n TXrxnAr 

^ IIICIC win UC a JDctUJV. UUllUll ailU a ivx\^iiu uuiiuil v/n an iiiw ^layiio ciiAvi luvivo. '^liwjviug, v*a 

button to bring the user to the previous screen (where it comes from) and on Menu button to bring 
the user to the first level button menu for another option. 

> There will be a Print button on each data report screen, where teachers can click to print the 
report(s). When they click the Print button, a window pops up and provides the choices of "Print 
All", "Print Current Student/Subtest", and "Print ..." 

• Print All: print all the reports of the form on the screen that the teacher is seeing. For example, 

if there arc a total of 35 reports (students), they will all be printed at one time. 

• Print Current Student/Subtest: print the current report of the form on the screen that the 

teacher is seeing. For example, if there are a total of 35 students' records but on the 
screen that is Jim's report, only Jim's is printed out. 

• Print . . there should be a pull-down menu with all the registered students' names for student 

reports or subtest names for subtest reports listed. Teacher can click to choose those 
he/she needs to print out. For example, the teacher can choose to print Jim, Lynn, 
and Susan's reports from 35 students. 

> Names on the X axis will be first initial + last name or full name (will be based on how many 
characters are acceptable for the database) 

> Text on the X and Y axes can change directions 

> For District/Region/School/Class data, the access permission will be as follows: 



Level 


Access 
Permission 


Grade level for comparison 


Kindergarteners 


First Graders 


Second Graders 


District access 


Districts 


✓ 


✓ 


✓ 


Regions 


✓ 


✓ 


✓ 


Schools 


✓ 


✓ 


✓ 




Region access 


Regions 




/ 


✓ 


Schools 


✓ 


/ 


✓ 




School access Schools / ^ 
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Formulas : 



Mean (Average): x == — JCi where n = observations (subject number) 
n 

Standard Deviation: SD = -1-^— {xi - xf 



Notes for Graphs and Tables: 

> Limited by space, the components of graphs are not proportionally sized. The layout of the graphs 
on screens should be designed such that 1) font for title should be bigger; 2) title should be placed 
in the center across top; 3) use right shapes / graphics for buttons, for example, right arrow for Go 
Next; 4) allow enough space between components on the screen; 5) everything on the screen 
should visually please eyes and be readable; and 6) use color but no more than 4 colors on each 
screen.) 

> For those graphs that cannot show all the data at a time (e.g., sample graphs 2 and 3 for class data), 
there will be next and previous arrow buttons, and zoom-m and zoom-out buttons. Default for 
these graphs will be the first 10 records. Click the "next" arrow button to show the next 10 records 
and "previous" arrow button to show the previous 10 records. Chck on the "zoom-out" button to 
see all the data, and "izoom-in" button to go back to default graph. 

> For those graphs showing subtests on the x axis, the test item numbers should be variables. They 
will change based on different versions (e.g., in the sample graph, there are 12 test items for 
Rhyming, the number 12 should change to 15 if there are 15 test items). 

> For all the bar graphs, there is a Data Values On/Off toggle button to provide the option of having 
or not having values on the graphs. 

> On some graphs and tables, there will be a cycle-through button (see samples). They can be 
clicked to cycle through the eight subtests, 6 grade level versions*, or the 35 students. (It will be 
replaced with a pull down / up menu for the user to select the items they'd like to have) 

> There should be somewhere on the screen to let the user know how to use the cycle-through 
buttons (or pull down/up menu) to see different graphs or tables, and toggle buttons to access 
different mode. 

> There will be a button on all the graph and table screens with the options of having a report 
displaying 1) all the test results, 2) Pre-test and the most recent post-test, and 3) the two most 
recent tests. (Definition: the first test is pre-test; the other tests are post-test ( 1 . . .n)) . 

> Display corresponding test dates for individual student's test result reports. 



* Kindergarten, First Grade, Second Grade, GradeKJFuU (full test), Gradel^Full, and Grade2_Full. 
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Class Data Graphs 

1 . Average score on all test sections (click on the subtest name (e.g.. Rhyming) to view more detailed . 
results for that subtest (graph 3)) 

For whatever version of the test (KA, KB, 1 A, IB, 2A, 2B, FA, FB), the graph layout will be designed 
the same way; all the subtests will be displayed. However, only those available subtests will be 
highlighted (active); all the unavailable subtests will be inactive (gray out). 



Average Score on P\\ Test Sections 
IVfe. Davis' Class 1Aat Central School 



m 



Re-Test ■ RDSt-Test 




DistrieR 
Region 
Rq>on Date 



Rtyrring 
(12lterre) 



First SoLTd 
Corrparison 
(12itenrB) 



Blending 
(12itenfS) 



Deletion 
(12ltaTB) 



LgtterNamB Letter Somd 
(26iterTC) (26iterT5) 



Ptnnetic 
Decoding 
(12itenB) 



Reading 
FliBTcy 
(40it€rrs) 




* This is an example of having data values on. All the other graphs are examples having data 
values off. 
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2, Total score comparison (Student names will display evenly across the X axis; Click on the student's 
name (e.g., J. Lambatos) to view individual student performance on all subtests (graph 4)) 




O 
c 

0) 



0) 



o 
o 



0) o 

o 

E 
o 
o 



Total Score Comparison 
A/Is. Davis' Class 1 A at Central School 



p Pre-Test ■ Post-Test 



District 



Report Date 




J. Lorrbatos LV\festy J. Rr2adeh ABelkin D. Susler C.Boyton 
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3. Students' performance on 1 sub-test (Student names will display evenly across the X axis; click on the 
student name to view the individual student's performance for all the subtests (graph 4); click on the ]^ 
button to see the corresponding Table: Comparison with Multi-State Sample Average. 




o 
w 

o 
o 

<D 
D) 
(0 

C 

o 
o 

0) 
Q. 



100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 



Rhyming 

Wfe. Davis' Class 1A at Centra! School 



I FYe-Test ■ Post-Test 



District 
Region 
Report Date 




J. Lonrbatos LV\A5sty J.Rrzadeh ABelWn D.Susler C.Boyton 
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4. Individual student performance o n all subtests 




Jack Lannbatos (age 6-11) - All Test Sections 
Ms. Davis, Qass 1A Oertral Schod 



I Re-Test ■Ftet-Test 



District 
Region 
Pre-test: Date 
Post-test: Date 




Rryrrirg First Soind Blerxfing Deletion Letter Nanre l£tterSoird FhDnetic Reading 
(12iterTs) Conp^son (12iterTB) (12iteTB) (26iteTB) (26iterTB) Decodrg Fliiancy 
(12iterT5) (12rterTB) (40itenB) 



X 



The above are sample Data Graph reports. Sub-test (e.g., Rhyming), and student name (e.g., J. Lambatos) 
are clickable. When they are clicked, another graph will display on the screen. The following flowchart and 
table show the hyperlink behavior and the number of graphs for each level: 

Hyperlink behavior 



The down arrows ^ 
show the flow 
through clicking 
on subtests or 
student names; the 
up arrows ^ show 
the flow through 
clicking on the 
back button. 



Graph 1 



Graph 3 


< ^ 


Table 





Graph 4 



Graph 2 



Graph 4 



The back button 
on graph 4 should 
be able to go back 
where it comes 
from, that means, 
to recognize 
whether it comes 
from graph 2 or 3. 
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Number of graphs on each level (Here number indicates the number of graphs for each individual single 
test version package) 



Graph (level) 


Number 


Name 


Graph 1 (level 1) 


1 


Average Score on All Test Sections 


Graph 2 (level 1) 


1 


Total Score Comparison 


Graph 3 (level 2) 


8 


Rhyming 






First Sound Comparison 






Blending 






Deletion 






Letter Names 






Letter Sounds 






Phonemic Decoding Accuracy 






Reading Fluency 


Graph 4 (level 2/3) 


35 (Max) 


Performance of Student name (age year - month) on All 
Test Sections 
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Class Comparison with M ulti-State Sample Average Table 

There will be a button at the first level named Class Comparison with Multi-State Sample Average. 
Clicking on this button will access an eight-button screen. Clicking on these buttons to access individual 
tables. The eight-button screen will include the following: 



Rhyming 


Letter Names 


First Sound Comparison 


Letter Sounds 


Blending 


Phonemic Decoding Accuracy 


Deletion 


Reading Fluency 



Clicking on one of the buttons (e.g.. Rhyming) to access the following table (Comparison with Multi-State 
Sample Average). The pre-test scores are sorted in descending order with the sample average and the 
header of Pre-Test highlighted as default. The Multi-State Sample Average row is always highlighted. To 
see post-test scores in descending order compared with sample average, click on the header Post-Test. Then 
the header of Post-Test is highlighted. 



Rhyming 

Comparison w^ith Multi-State Sample Average 
Ms, Davis' Class 1 A at Central School 



District 
Region 
Rq)OTt Date 





Student 




Post-Test 


1 


Jack. Lambatos 


15 


14 


2 


Leighton Pirzdeh 


15 


15 


3 


Joe Susler 


14 


15 


4 


Anna Boyston 


13 


13 


5 


David Westy 


11 


14 










6 


Christy Belkin 


8 


11 











^^^^ ^^ifgi^i 




• To see post-test scores in descending order compared with sample average, click on the header Post-Test. 

• Qick on each student's name to access Individual Student's Data Summary. 



^ Based on performance of n = 600 from CA, IL, FL. 
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Individual S tudent Data Summary 

This report can also be accessed through the "Individual Student Data Summary" button on the "first level 
button menu" screen. This report includes two test levels (grade level and full test) for each individual 
student. Items correct of the test taken, including test number and percentage, are shown in the table. 



Test Report for Melissa (Kindergarten, Age 5-3) 



District 
Region 
School 
Rqx)rt Date 



Test Sections 


Number 
of Test 
Items 


Items Correct 


J est oampie 
Average Scores^ 


Pre-Test 


Post-Test 


Date 


Date 


Kindergarten version 




No. 


% 


No. 


% 


Sample size = 


Rhyming 


12 


12 


. 100% 






Average = 


First Sound Comparison 


12 


9 


75% 






Blending 


12 


6 


50% 






Deletion 


12 


9 


75% 






Letter Names 


26 


20 


77% 






Letter Sounds 


26 


20 


77% 






Full version 




Date 


Date 


Sample size F = 
Sample size KF = 
Sample size 1F = 
Sample size 2F =^ 


No. 


% 


No. 


% 


Rhyming 


12 










Average F = 
Average KF = 
Average IF = 
Average 2F =^ 


First Sound Comparison 


12 










Blending 


12 










Deletion 


12 










Letter Names 


26 










Letter Sounds 


26 










Phonemic Decoding Accuracy 


12 










Reading Fluency 


40 











' Based on preliminary data gathered from sample (n = 450) in TN, OH, FL, and IL. 
^ Sample size F = (sample size of all those who took Full version); Sample size KF = (sample size of all K 
graders who took Full version); Sample size IF = (sample size of all first graders who took Full version); 
Sample size 2F = (sample size of all second graders who took Full version); For a student report, only 
Sample size F and the sample size for that student's grade (e.g., for kindergarten student report, show 
Sample size F and Sample size KF) need to be shown in the report. 

^ Average F = (Average score of all those who took Full version); Average KF = (Average score of all 
those K graders who took Full version); Average IF = (Average score of all those first graders who took 
Full version); Average 2F = (Average score of all those second graders who took Full version); For a 
student report, only Average F and the average for that student's grade (e.g., for kindergarten student 
report, show Average F and Average KF) need to be shown in the report. 
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Test Report for Juanita (First Grade, Age 6-10) 



District 
Region 
School 
Rq}OTt Date 



Test Sections 


Number 


Items Correct 


Test Sample 




of Test 


Pre-Test 


Post-Test 


Average Scores* 




Items 


Date 


Date 




First grade version 




No. 


% 


No. 


% 


ni = (sample size) 


Blending 


12 


10 


80% 






Xi = 


Deletion 


12 


12 


100% 








Phonemic Decoding Accuracy 


12 


9 


75% 








Reading Fluency 


40 


30 


75% 










Full version 




Date 


Date 


np, nicF,niF,n2F^ 






No, 


% 


No. 


% 


Rhyming 


12 












First Sound Comparison . 


12 










Blending 


12 










Xkf = 


Deletion 


12 










XlF = 


Letter Names 


26 












Letter Sounds 


26 












Phonemic Decoding Accuracy 


12 












Reading Fluency 


40 













^ Based on preliminary data gathered from sample (n = 600) in CA, FL, and IL. 

^ np = (sample size of all those who took Full version); njcF = (sample size of all K graders who took Full 
version); njp = (sample size of all first graders who took Full version); = (sample size of all second 
graders who took Full version) 
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District 

Test Report for Tyron (Second Grade, Age 7-9) ^^^^ 

Rq)OTt Date 



Test Sections 


Number 


Items Correct 


1 est oampie 




of lest 


Pre-Test 


Post-Test 


/werage ovurea 




Items 


Date 


Date 




Second grade version 




No. 


% 


No. 


% 


n2 = (sample size) 


Deletion 


12 


9 


75% 








Phonemic Decoding Accuracy 


12 


6 


50% 






Reading Fluency 


40 


32 


80% 








Full version 




Date 


Date 


Up, nKF,niF,n2F^ 






No. 


% 


No. 


% 


Rhyming 


12 










Xf = 


First Sound Comparison 


12 












Blending 


12 










Xkf~ 


Deletion 


12 












Letter Names 


26 












Letter Sounds 


26 










X2F- 


Phonemic Decoding Accuracy 


12 












Reading Fluency 


40 













' Based on preliminary data gathered from sample (n = 600) in CA, FL, and IL. 

^ Up = (sample size of all those who took Full version); nKF = (sample size of all K graders who took Full 
version); n,F = (sample size of all first graders who took Full version); n2F = (sample size of all second 
graders who took Full version) 
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The above are sample Data Tables and Summary reports. Clicking on students' names and the cycle- 
through buttons will move to another report. The following flowchart and table show the hyperlink 
behavior and the number of reports for each level: 

Hyperlink behavior 



The down arrows ^ 
show the flow 
through clicking 
on subtest or 
student names; the 
up anrows ^ show 
the flow through 
clicking on the 
back button. 



Eight-button screen 



Students' Performance on Each subtest, 
comparison v/ith sample average 



Individual student data summary 



^ ^ Cycle through eight subtests 



^ ^ Cycle through 35 students (Max) 



Number of reports on each level (Here number indicates the number of graphs for each individual single 
test version package) 



Graph (level) 


Number 


Name 


Eight button level 


1 


N/A 


Report level 1 


8 


Rhyming: Comparison with Multi-State Sample Average 






First Sound Comparison: Comparison with Multi-State 
Sample Average 






Blending: Comparison with Multi-State Sample Average 






Deletion: Comparison with Multi-State Sample Average 






Letter Names: Comparison with Multi-State Sample 
Average 






Letter Sounds: Coniparison with Multi-State Sample 

Average 






Phonemic Decoding Accuracy: Comparison with Multi- 
State Sample Average 






Reading Fluency: Coniparison with Multi-State Sample 

Average 


Report level 2 


35 (Max) 


Test Report for Student name {age year-month) 
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District/Region/School/Class Data Graphs & Tables 

When the "Comparison of Regions/Schools/Classes to Multi-State Sample Average" button on the "first 
level button menu" screen is clicked, districts can directly access the following graph 1 ; regions can 
directly access the following graph 2; and schools can directly access the following graph 3. Lower level 
graphs can also be accessed through higher level graphs. 



1 . For School Districts (click the region name to access the graphs for all the schools' data; click the bars 
to access database tables for pre-test/post-test scores, where teachers can use sort and search functions 
to sort/search region information needed) 





Regions Comparison 
First Grade Averages, District 




a Pre-Test 
B Post-Test 



Region 1 



Region 2 



Region 3 



Region 4 
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2. For Regions (click the school name to access the graphs for all the school classes' data - three separate 
graphs of for Kindergarteners, for First Graders, and for Second Graders; click the bars to access 
database tables for pre-test/post-test scores, where teachers can use sort and search functions to 
sort/search school information needed) 




Schools Comparison 
First Grade Averages, Region, District 



(0 

a> 
1- 

o 
o 

CO 
o 
o> 

(9 
I. 

> 
< 




I Pre-Test 
I Post-Test 



School 1 School 2 School 3 School 4 



^^^^^ ^^^^^^ ^^^^^ ^gl^^^^g^^gl ^^ffi!(^ass^^*4a^i!^^ 
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3. For Schools (click the Performance Details button to access For School Performance Details graph; 
click the bars to access database tables for pre-test/post-test scores, where teachers can use sort and 
search ftinctions to sort/search class information needed) 




Comparison of Rrst-Grade Classes' Total Score Averages 
School, Region, [Xstrict 




aPre-Test 
9 Post-Test 



Oassl 



aass2 



Qass3 



aass4 



imjJsiMuaBmiJiuiuu.u»]o>[iL i i 

DWl5aBels"On/Off^-=' 

■ ^^^^^S^^^^^^^^^^g ^^^^^^^^^^^ 



4. For School Perfonnance Details (the test item numbers should be variables. They will change based on 
different versions (e.g., in the sample graph, there are 12 test items for Rhyming, the number 12 will 
change to 15 if there are 15 test items)). 



First-Grade Average Scores on All Test Sections 
School, Region, District 



iPre-Test ■ Post-Test 




Rhyning First Sound Blending Deletion Letter Nan^ Letter Sound Ptwnetic Reading 
(12iteaB) Conparison (12 items) (12 iteiTB) (26 iterre) (26 items) Decoding Fluency* 
(12 iteiTB) (12 iteiTE) (40 itenre) 
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The above are sample Data Graph reports at district, region, and school levels. The following flowchart and 
table show the hyperlink behavior and the number of reports for each level: 

Hyperlink behavior 



The down arrows ^ 
show the flow 
through clicking 
on corresponding 
buttons; tiie up 
arrows ^ show the 
flow through 
clicking on the 
back button. 



For School Districts 



For Regions 



For Schools 



1 



^ ^ Cycle through six grade versions 



^ ^ Cycle through six grade versions 



^ ^ Cycle through six grade versions 



For School Performance Details ^ ^ Cycle through six grade versions 



Number of reports on each level 



Graph (level) 


Number 


Name 


For school districts (level 1) 


6 


Regions Comparison: Kindergarten Averages, District 






Regions Comparison: First Grade Averages, District 






Regions Comparison: Second Grade Averages, District 






Regions Comparison: GradeK_Full Averages, District 






Regions Comparison: Grade 1 Full Averages, District 






Regions Comparison: Grade2_Full Averages, District 


For regions (level 2) 


6 


Schools Comparison: Kindergarten Averages, Region, 
District 






Schools Comparison: First Grade Averages, Region, 
District 






Schools Comparison: Second Grade Averages, Region, 
District 






Schools Comparison: GradeK_Full Averages, Region, 
District 






Schools Comparison: Grade l_Full Averages, Region, 
District 






Schools Comparison: Grade2_Full Averages, Region, 
District 


For schools (level 3) 


6 


Classes Comparison: Kindergarten Averages, School, 
Region, District 






Classes Comparison: : First Grade Averages, School, 
Region, District 






Classes Comparison: Second Grade Averages, School, 
Region, District 



Copyright © 2000 Cognitive Concepts, Inc.: All rights reserved 
Instructional Designer: Feng-Qi Lai, Ph.D. 



OPAL Data Graphing .doc. Page 17 of 24, 6/2/01 

A-n 



OPAL Data Graphing 



fli^jg Cognitive 

Concepts, inc.'" 

Solutions for Literacy and Language'' 







Classes Comparison: GradeK Full Averages, School 
Region, District 






Classes Comparison: Grade l_Full Averages, School, 
Region, District 






Classes Comparison: Grade2_Full Averages, School, 
Region, District 


For school performance details 
(Level 4) 


6 


Kindergarten Average Scores on All Test Sections 

School, Region, District 






First Grade Average Scores on All Test Sections 
School, Region, District ' 






Second Grade Average Scores on All Test Sections 

School, Region, District 






GradeK Full Average Scores on All Test Sections 
School, Region, District 






Gradel_Full Average Scores on All Test Sections 
School, Region, District 






Grade2_Full Average Scores on All Test Sections 
School, Region, District 



Comparison with multi-state sample averages across Regions/Schools/Classes 

There will be a button at the first level named Comparison of Regions/Schools/Classes to Multi-State 
Sample Average. Clicking on this button will access a three-button screen. Clicking on these buttons to 
access individual tables. The three-button screen will include the following: 




Comparison across Regions 



Comparison across Schools 



Comparison across Classes 



When districts access to this screen all the buttons are active; when regions access this screen. Comparison 
across Regions are inactive; When schools access this screen, only Comparison across Classes button is 
active. Clicking on a button to access the following table accordingly. The default table will be Comparison 
on Rhyming. The pre-test scores are sorted in descending order with the sample mean and the header of 
Pre-Test highlighted as default. To see post-test scores in descending order compared with sample mean, 
click on the header Post-Test. Then the header of Post-Test is highlighted. Click on individual 
Region/school can access this region/school's Comparison with Multi-State Sample Average across 
Schools/Classes table. 
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Rhyming 

Comparison of Regions to Multi-State Sample Average 
Grade 1, Full, Dwrricr 



Report Date 





Region 


N 


Mean Percentage Correct 




Post-Test 


1 


Region 3 


23 


90% 


92% 


2 


Region 5 


26 


88% 


88% 


'> 
J 


T> — : — '1 




86% 


90% 


4 


Region 1 


30 


86% 


89% 




Region 6 


25 


79% 


79% 












6 


Region 4 


26 


77% 


79% 













• To see post-test scores in descending order compared with sample average, click on the header Post-Test. 

• Click on each region *s name to access Comparison of Schools to Multi-State Sample Average. 



Rhyming 

Comparison of Schools to Multi-State Sample Average 
Grade 1, Full,jRegto«, District 



Report Date 





School 


N 


Mean Percentage Correct 




Post-Test 


1 


School 3 


23 


90% 


92% 


2 


School 5 


26 


88% 


88% 


3 


School 2 


22 


86% 


90% 


4 


School 1 


30 


86% 


89% 


5 


School 6 


25 


79% 


79% 












6 


School 4 


26 


77% 


79% 













• To see post-test scores in descending order compared with sample average, click on the header Post-Test. 

• aick on each school's name to access Comparison of Classes to Multi-State Sample Average. 



Based on performance of n = 450 from CA, IL, FL. 



Copyright © 2000 Cognitive Concepts, Inc.: All rights reserved 
Instructional Designer: Feng-Qi Lai, Ph.D. 



OPAL Data Graphing .doc, Page 19 of 24, 6/2/01 



OPAL Data Graphing 




Cognitive 
Concepts, inc. 



Solutions for Literacy avd Language^ 



Rhyming 

Comparison of Classes to Multi-State Sample Average 
Grade 1, VuMj School, Region, District 



Report Date 





Class 


N 


Mean Percentage Correct 










Post-Test 


1 


Class 3 


23 


90% 


92% 


2 


Class 5 


26 


88% 


88% 


3 


Class 2 


22 


86% 


90% 


4 


Class 1 


30 


86% 


89% 


5 


Class 6 


25 


79% 


79% 












6 


Class 4 


26 


77% 


79% 














To see post-test scores in descending order compared with sample average, click on the header Post-Test. 



The above are sample Data Table reports at district, region, and school levels. The following flowchart and 
table show the hyperlink behavior and the number of reports for each level: 

Pypgriink behavior 



The down arrows ^ 
show the flow 
through clicking 
on region/school 
names; the up 
arrows show the 
flow through 
clicking on the 
back button. 



Across regions 



I 



Across schools 



Across classes 



Cycle through eight subtests 
and six grade versions 

Cycle through eight subtests 
and six grade versions 



Cycle through eight subtests 
and six grade versions 



Based on performance of n = 450 from CA, IL, FL. 
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Number of reports on each level 







N^nie 


For school districts (level 1) 


8x6 


Rhyming: Comparison of Regions to Multi-State Sample 
Average 






First Sound Comparison: Comparison of Regions to 
Multi-State Sample Average 

Kjraae version, uisinci 






Blending: Comparison of Regions to Multi-State Sample 
Average 

Kjruus versiOfi, JLJUiiriLi 






Deletion: Comparison of Regions to Multi-State Sample 
Average 

Grade version^ District 






Letter Names: Comparison of Regions to Multi-State 
Sample Average 

Grade version, District 






Letter Sounds: Comparison of Regions to Multi-State 
Sample Average 

Grade version. District 






Phonemic Decoding Accuracy: Comparison of Regions 
to Multi-State Sample Average 

Grade version. District 






Reading Fluency: Comparison of Regions to Multi-State 
Sample Average 

Grade version. District 


For regions (level 2) 


8x6 


Rhyming: Comparison . of Schools to Multi-State Sample 
Average 

\jfuiic ycr^iun, jxc^n/ft, x./(<jC/»c( 






First Sound Comparison: Comparison of Schools to 
Multi-State Sample Average 

Grade version. Region, District 






Blending: Comparison of Schools to Multi-State Sample 
Average 

Grade version. Region, District 






Deletion: Comparison of Schools to Multi-State Sample 
Average 

Grade version. Region, District 






Letter Names: Comparison of Schools to Multi-State 
Sample Average 

Grade version. Region, District 
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Letter Sounds: Comparison of Schools to Multi-State 
Sample Average 

Grcidc version^ Region, District 






Phonemic Decoding Accuracy: Comparison of Schools 
to Multi-State Sample Average 

Grade version. Region, District 






Reading Fluency: Comparison of Schools to Multi-State 
Sample Average 

Grade version. Region, District 


For schools (level 3) 


Sx6 


Rhyming: Comparison of Classes to Multi-State Sample 

Average 

Grade version, School, Region, District 






First Sound Comparison: Comparison of Classes to 
Multi-State Sample Average 

Grade version. School, Region, District 






Blending: Comparison of Classes to Multi-State Sample 
Average 

Grade version. School, Region, District 






Deletion: Comparison of Classes to Multi-State Sample 

Average 

Grade version, School, Region, District 






Letter Names: Comparison of Classes to Multi-State 
Sample Average 

Kjrciuc Version, ocnooi, ixcgion, uisinci 






Letter Sounds: Comparison of Classes to Multi-State 
Sample Average 

Grade version. School, Region, District 






Phonemic Decoding Accuracy: Comparison of Gasses 
to Multi-State Sample Average 

Grade version. School, Region, District 






Reading Fluency: Comparison of Classes to Multi-State 
Sample Average 

Grade version. School, Region, District 
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Raw Scores and Response Time for Each Student (This design is for CCI research - data analysis) 



Student ID 


Test Item M 


Response' 


c(i)/i(or 


R(1)/NR(0)' 


Response Time (sec)* 


Time Run-ouf" 


Student 1 


3 


1 


1 


2 


0 


Student 2 


9 


0 


1 


40 


1 


Student 3 


4 


0 


0 


2 


0 


Student 4 


3 


1 


0 


3 


0 . 


Student 5 


2 


0 


0 


1 


0 


Student 6 


9 


0 


1 


35 


1 


Student 7 




1 


0 




0 


Student 8 


2 


0 


1 


20 


0 


Student 9 


3 


1 


1 


25 


1 



. Statistics 



Mean correct 


Standard Deviation 


Mean Time 


Other 


> Individual student 
test mean for each 
test item category 


> Standard Deviation for 

Individual student test of each 
test item category 


Mean time for Individual 
student test of each test 
item category 


> 


> Individual student 
subtest mean 


> Standard Deviation for 
Individual student subtest 


Individual student subtest 
mean time 




> Individual student 
total test mean 


> Standard Deviation for 
Individual student total test 


Individual student total 
test mean time 




> Group mean for each 
test item* 


> Standard Deviation for each 
test item in a group 


Group mean time for each 
test item 




> Group mean for each 
test item category 


> Standard Deviation for each 

test item category in a group 


Group mean time for each 

test item category 




> Group mean for each 
subtest 


> Standard Deviation for each 
subtest in a group 


Group mean time for each 
subtest 




> Group mean for total 
test 


> Standard Deviation for total 
test in a group 


Group mean time for total 
test 





' One (1) to six (6) are used as students' response to test items; e.g., there are four choices for the test items 
in subtest one. They will be labeled as 1 to 4, If the student clicks the third picture, it will be recorded as 
3. Nine (9) is recorded as no response. 

^ One ( 1) for correct answer and 0 for incorrect answer. 

^ One (1) for having presented instructions twice and 0 for having presented instructions only once. 

^ When there is no response for 20 seconds, instructions will be repeated automatically; if there is no 
response for another 20 seconds, the screen flips to the next test item. No response will be coded as 0 
(incorrect) and time 40 seconds (see student 2). Instructions can only be repeated once. If the student 
clicks repeat button but does not answer the question for 20 seconds from the repeated instruction, the 
screen will flip to the next test item (see student 6). 

^ One (1) for time run-out (see student 2) and 0 for student spending 20 seconds to answer the question 
before the instruction is repeated (see student 7). 

* Group mean for each test item will be for internal analysis only 
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Access to data 

> CCI has secured access to all the data recorded (need statement of security and permission) 

> School districts have secured access to their school district students' data 

> Regions have secured access to their region school students' data 

> Schools have secured access to their school students' data 

> Class teachers have secured access to their class students' data 

> Parents have secured access to their own children's data 
Other 

> Include an On/Off option for limiting time allowed for user responses* 

^ Include an On/Off option for limiting number of times instructions repeated^ 




Add disclaimers: Turning off limited time for response and/or limited repetition of instructions prevents 
valid assessment of skills and comparison of scores across subjects and comparison of pre-test vs. post-test. 
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Subtest 4: Blending 
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Subtest 2: Rhyme Generation 
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Error Measure 



Figure 2 A 



f Stot ^ ^lO 

J 



i= 1 



7=1 



/=1 



1 



9lG 



Compare Incorrect Response 
with Error Measure 



3l<> 



y=y + i 



; = ; + l 



No 



No 



No 




Label error 



Store in database 



(ST),: Subtest; /■= 1 ... k 

(IR),y: Incorrect Response; ij= 1.1, 1.2,...., 1. max., ... k.l, k.2, k.max. (/ = 1 ... 

max.) 

(EM),/: Error Measure; //= 1.1, 1.2, l.max., ...k.l, k.2, kmax. (/= 1 ... max. 
Note: max. for / is different from max. for j.) 
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6. Maoipilating 


©1 
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l©l ^ 
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7. Recalling in 
Sequential Order 


©1 


1©I 








\ ^ i 


8. Rapid Naming 


©1 


i©l 


i®l\l( 


Dl- 




1 ^ \ 


9, Letto Nmws «nd i 
Sound i 


m 


/|0L-^'^ 


T)|... 




\ Xh. \ 


10. Decoding | 




'1©I i( 


7)1... 




\ ^ 1 


11. Fluent Reading I (^^\ 


i©i 


J©1 1( 









Subtest 


Error Measure ID 


Error Measure 


3. Beginning and Ending 
Sounds' 


2 


Do not recognize Ifl when it is at the end 
following an l\l sound 


4. Blending 


3 


Can not blend when /fi^ is at the end following an 
Id sound 


5. Segmenting 


4 


Do not count Ifl when it is at the end following an 
sound 


6. Manipulating 


4 


Can not replace If! with l\l when it is at the end 
following an Id sound 


7. Recalling in Sequential 
Order 


3 


Do not recall Ifl when it is at the end following 
another consonant. 


9. Letter Names and Sound 


4 


Can not recognize Ifl sound when it is isolated 


10. Decoding 


2 


Cannot recognize the nonsense word when the 
word ends with a IfJ sound which follows another 
consonant 



vo 
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Figure 4 
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180 
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CALCULATE SCORE 
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END 
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